Neuromorphic Computing-Assisted Triboelectric Capacitive-Coupled Tactile Sensor Array for Wireless Mixed Reality Interaction

Flexible tactile sensors show promise for artificial intelligence applications due to their biological adaptability and rapid signal perception. Triboelectric sensors enable active dynamic tactile sensing, while integrating static pressure sensing and real-time multichannel signal transmission is key for further development. Here, we propose an integrated structure combining a capacitive sensor for static spatiotemporal mapping and a triboelectric sensor for dynamic tactile recognition. A liquid metal-based flexible dual-mode triboelectric-capacitive-coupled tactile sensor (TCTS) array of 4 × 4 pixels achieves a spatial resolution of 7 mm, exhibiting a pressure detection limit of 0.8 Pa and a fast response of 6 ms. Furthermore, neuromorphic computing using the MXene-based synaptic transistor achieves 100% recognition accuracy of handwritten numbers/letters within 90 epochs based on dynamic triboelectric signals collected by the TCTS array, and cross-spatial information communication from the perceived multichannel tactile data is realized in the mixed reality space. The results illuminate considerable application possibilities of dual-mode tactile sensing technology in human–machine interfaces and advanced robotics.


S3
Supporting Information Note S1 In the initial state without applied pressure, the contact points of the upper and lower silicone rubber dielectric layers are defined as the origin of the coordinates and the plane right-angle coordinates are established as shown in Figure S1.The arch shaped part of the sensing unit can be approximated as a hemispherical structure, where r is the radius of the hemisphere.A point m (x,d) on the surface of the upper silicone rubber layer is taken as an example in the initial state.When pressure acts on the sensing unit, the displacement of point m in the vertical direction is defined as y.In this case, the vertical distance between the upper and lower silicone rubber layers at point m can be expressed as: The area of the sensing unit surface in the arch shaped part at the height where the point m (,) located can be expressed as The total capacitance between two electrodes could be equated to the series capacitance of air and silicone rubber, both of which performing as the dielectric media.The formula is presented as: where   is the capacitance of silicone rubber and   is the capacitance of the air between two electrodes.
In this case, the specific capacitance expressions of   and   are presented in the following: - 2 - 2 - where  0 is the vacuum permittivity,   is the relative permittivity of silicone rubber, S is the effective overlapping area between two electrodes and  0 is the thickness of silicone rubber dielectric layer.

Figure S1 .
Figure S1.Optical photograph of the TCTS array: (A) Side and (B) front view of the up panel.(C) Top view of the down panel.(D)Top view of the whole device before and (E) after stretching to 140%.(F) TCTS array put on the 3D printed foot model.

Figure S2 .
Figure S2.Fabrication process of the TCTS array.

Figure S3 .
Figure S3.Simulation showing the deformation process of the TCTS unit under mechanical pressure of 15.34 kPa.

Figure S4 .
Figure S4.Equivalent circuit diagram of the TCTS unit composed of a load capacitance connected with a single-electrode-mode TENG.

Figure S5 .
Figure S5.A cycle of electricity generation process for illustrating the working mechanism of the single-electrode mode TENG.

Figure S6 .
Figure S6.Surface potential simulation during an operation cycle process of the single-electrode mode TENG.

Figure S8 .
Figure S8.Baseline noise voltage of the triboelectric sensor.

Figure S10 .
Figure S10.Relationship between output voltage and applied pressure for 16 units of TCTS array.

Figure S11 .
Figure S11.Cross-talk influence.(A) Schematic diagram and (B) optical photographs of the sensing array with labelled units.(C) Comparison in output voltage of surrounding unit No. 2, (D) No. 5, (E) No. 7 and (F)No.10 before and after unit No. 6 compressed at the highest load of 78 kPa.

Figure S12 .
Figure S12.Output voltage curves of different contact materials under the pressure of 15.34 kPa.

Figure S15 .
Figure S15.The pressure sensitivity of the capacitive sensor.

Figure S16 .
Figure S16.The PPF index indicating short-term plasticity (STP) characteristics of the synaptic transistor.

Figure S17 .
Figure S17.The EPSC behavior of the synaptic transistor.

Figure S19 .
Figure S19.The EPSC and IPSC results of the synaptic transistor during one cycle.

Figure S20 .Figure S21 .
Figure S20.LTP/D characteristic curves as a function of number of pulses.

Figure S22 .
Figure S22.Designed SLP-based ANN with a size of 2501× 10 for the training and recognition simulation.The enlarged view demonstrates the synaptic weight presented by the conductance difference of two equivalent synaptic devices.

Figure S23 .Figure S24 .
Figure S23.Recognition rate curves of handwriting number, handwriting letter and force strength.

Figure S25 .
Figure S25.Flow chart of mixed reality interface application in Unity.

Table S1 .
Statistical results of the electrical output and accuracy analysis affected by cross-talk.

Table S2 .
Comparison on key metrics of recent triboelectric and capacitive tactile sensor arrays.